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BACKGROUND OF THE INVENTION 



(1) Field of the Invention 

The present invention relates to the field of computer graphics. In 
5 particular, the present invention relates to a system and a method for detecting 
human faces in an existing color graphics image. 

(2) Description of Related Art 

An automatic face recognition system should have the ability to identify 
one or more persons in a scene, starting from still or video images of the scene. 

10 A complete solution of the problem involves segmentation of faces from 

cluttered scenes, extraction of features from the face region, identification and 
matching. The first step in this process is face detection. The goal of face 
detection is to determine whether there is one or more human faces in the 
image, and, if present, return its location and spatial extent for further 

15 processing. 

Most of the existing face detection systems use window-based or pixel- 
based operation to detect faces. In a window-based system, a small window is 
moved over all portions of an image to determine whether a face exists in each 
window based on distance metrics. Common problems with window-based 

20 approaches are that they cannot detect faces of different orientations or view 
angles, and that they are computationally expensive. 

In representative pixel-based analyses, color segmentation is used as the 
first step. This assumes that human skin colors fall in a small, known region 
in color space. Some advantages of using color as feature are speed and 

25 invariance to orientation. However, it is difficult to model human skin color for 
several reasons. One reason is that different cameras produce significantly 
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different color values. Another reason is that human skin color differs from 
person to person. Finally, skin color distribution shifts with changes in 
ambient illumination. 

Therefore a method for reliably and efficiently detecting human faces 
within a graphical image is at issue in face recognition systems. 
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SUMMARY 



A system and method for determining a series of candidate patches for 
human faces in a color graphic image is disclosed. The method may start by 
5 determining a first area wherein a color gradient has a low value, and then 
determining a second area wherein an intensity value has a high value. The 
method may then perform a logical AND on the first area and the second area 
to create a third area. The method may then select portions of this third area 
with suitable hue saturation values to form a series of candidate patches where 
10 the human faces reside. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



The features, aspects, and advantages of the present Invention will 
become more fully apparent from the following detailed description, appended 
5 claims, and accompanying drawings in which: 

Figure 1 is a block diagram of a computer graphics system, according to 
one embodiment of the present invention; 

Figure 2 is an unprocessed graphical image, according to one 
embodiment of the present invention; 
10 Figure 3 is a color gradient map of the image of Figure 2, according to 

^ one embodiment of the present invention; 

Figure 4 is an intensity map of the image of Figure 2, according to one 
*J embodiment of the present invention; 

W Figure 5 is a combined map of the color gradient map of Figure 3 and 

* 15 the intensity map of Figure 4, according to one embodiment of the present 
01 invention; 

r.l Figure 6 is a hue spectrum chart, according to one embodiment of the 

S present invention; 

Figure 7 is a set of candidate patches and determined ellipses for the 
20 image of Figure 2, according to one embodiment of the present invention; 

Figure 8 shows the face candidate map superimposed on the image of 
Figure 2, according to one embodiment of the present invention; 

Figure 9 shows the final result of the process, according to one 
embodiment of the present invention; and 
25 Figure 10 is a flowchart of process steps, according to one embodiment 

of the present invention. 
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DETAILED DESCRIPTION 



In the following description, numerous specific details are set forth to 
provide a thorough understanding of the present invention. However, one 
5 having an ordinary skill in the art may be able to practice the invention without 
these specific details. In some instances, well-known circuits, structures, and 
techniques have not been shown in detail in order not to unnecessarily obscure 
the present invention. 

Psychological studies of human face recognition by humans suggest that 

10 virtually every kind of available information is used simultaneously. For 
example, the configuration of the facial features can help humans find the 
location of the face, since it is known that the features can not appear in 
arbitrary arrangements. In contrast, a computer graphics system cannot use 
all kinds of information simultaneously. A limited number of kinds of 

15 information must be processed at any given time. Important constraints to the 
processing of selected kinds of information include simplicity of estimation of 
parameters, low dependency upon ambient light intensity, low dependency 
upon small changes of facial expression, and maximizing information content 
in those kinds of information selected. 

20 In one embodiment of the present invention, color, shape, intensity, and 

face configuration information are combined together to locate faces in color 
images. This method of face detection advantageously is computationally 
efficient. 

Referring now to Figure 1, a block diagram of a computer graphics 
25 system 100 is shown, according to one embodiment of the present invention. 
Computer graphics system 100 may include read-only memory (ROM) 102, 
random-access memory (RAM) 104, one or more central processing units (CPU) 
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106, one or more graphics controllers 108 with attached displays 1 10, 
connections to a local area network (LAN) 120 and wide area network (WAN) 
122 via LAN controller 112 and WAN controller 114, respectively, mass storage 
devices 1 16, and removable media 1 18. The functional parts of computer 
5 graphics system 100 may be connected via one or more system busses 124. 
Software executing on computer graphics system 100 may be loaded via the 
LAN 120, the WAN 122, or the removable media 1 18. 

Referring now to Figure 2, an unprocessed graphical image 200 is shown, 
according to one embodiment of the present invention. Exemplary graphical 

10 image 200 includes five faces, 202, 204, 206, 208, and 210. Detecting these 
faces and exhibiting their size, shape, and location will be discussed in 
conjunction with Figure 2 and related figures, Figures 3, 4, 5, 7, 8, and 9 
below. In one embodiment, the following four assumptions are made: the 
facial area is relatively smooth and bright; the only lighting is ambient 

15 illumination; the outlines of human faces are roughly elliptical; and there are 
significant color and intensity changes inside facial areas because of the 
configuration of facial features. 

The image of Figure 2 is represented by a large number of picture 
elements (pixels), each with three words of data representing color and 

20 intensity. The words in one embodiment may include 8 bits. In alternate 

embodiments other word lengths may be used. In one embodiment, the three 
words may correspond to red intensity, green intensity, and blue intensity 
(RGB format). In alternate embodiments, the three words may be separated in 
other color spaces, such as luminance, chrominance minus blue, chrominance 

25 minus red (YCC format). 

Referring now to Figure 3, a color gradient map 300 of the image of 
Figure 2 is shown, according to one embodiment of the present invention. In 
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one embodiment, the color gradient is the rate of change of color when going 
from one area to another. The color gradient may be used to extract 
homogeneous regions since it may be assumed that the facial area is relatively 
smooth. The color gradient may also extract edge information. 
5 In order to calculate the color gradient, various methods of calculation 

may be used. In one embodiment, a Sobel filter, well-known in the art, is used 
to calculate a map of the magnitude of the color gradient. Commercially 
available Sobel filter software is available for many common computer graphics 
workstations. 

10 The map of the magnitude of the color gradient, determined immediately 

above, is then converted by applying a true or false threshold value into a map 
wherein each pixel is represented by a single bit of data, true or false. The 
threshold is determined by the process of normalization. In normalization, the 
average value of the magnitude of the color gradient of all the pixels is 

15 determined. A fixed percentage of the average value is selected as the 
threshold, and then the magnitude of the color gradient for each pixel is 
compared with the threshold. If the magnitude is less than the threshold, the 
pixel is considered a "true". If the magnitude is more than the threshold, the 
pixel is considered a "false". 

20 In one embodiment, the map wherein each pixel is represented by a 

single bit of data, determined as in the paragraph immediately above, is 
converted into a simplified mosaic representation. Use of a simplified mosaic 
representation filters out any noise yet retains the facial information. In one 
embodiment, each group of 4 by 4 pixels is replaced by a single mosaic pixel 

25 with a value assigned by determining the predominant value, true or false, 

assigned to the 16 pixels within the mosaic pixel. In other embodiments, other 
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sizes of mosaic blocks may be used. This simplified mosaic representation of 
the above-described map may be termed a color gradient map. 

Figure 3 is a representative color gradient map for the image of Figure 2, 
derived in accordance with the steps outlined above. The white area 310 is a 
5 region wherein the mosaic pixel values are all true. The black area 320 is a 
region wherein the mosaic pixel values are all false. In accordance with one 
embodiment, the black area 320 are areas rejected as candidate face locations, 
whereas the white areas 310 are areas considered for further processing as 
candidate face locations. 

10 Referring now to Figure 4, an intensity map 400 of the image of Figure 2 

is shown, according to one embodiment of the present invention. It is assumed 
that any faces in the image are well iUuminated and well focused. Therefore, 
the facial areas may have higher intensities than the surrounding areas. 
Again, a threshold value may be determined by the process of 

15 normalization. An intensity value for each pixel is first determined. An average 
value of the intensity values for all the pixels is then determined. A percentage 
of this average value is then selected as a threshold value. 

The intensity value of each pixel is then compared with the threshold 
value. If the intensity of a pixel is greater than the threshold value, the pixel is 

20 assigned a value "true". If the intensity of a pixel is lower than the threshold 
value, the pixel is assigned a value "false". 

As in the case of the color gradient map derivation, in one embodiment 
the map wherein each pixel is represented by a single bit of data, determined 
as in the paragraph immediately above, is converted into a simplified mosaic 

25 representation. In one embodiment, each group of 4 by 4 pixels is replaced by 
a single mosaic pixel with a value assigned by determining the predominant 
value, true or false, assigned to the 16 pixels within the mosaic pixel. In other 



80398.P322 



9 



embodiments, other sizes of mosaic blocks may be used. This simplified 
mosaic representation may be termed an intensity map. 

Figure 4 is a representative intensity map for the image of Figure 2, 
derived in accordance with the steps outlined above. The white area 410 is a 

5 region wherein the mosaic pixel values are all true. The black area 420 is a 
region wherein the mosaic pixel values are all false. In accordance with one 
embodiment, the black area 420 are areas rejected as candidate face locations, 
whereas the white areas 410 are areas considered for further processing as 
candidate face locations. 

10 Referring now to Figure 5, a combined map 500 of the color gradient map 

of Figure 3 and the intensity map of Figure 4 is shown, according to one 
embodiment of the present invention. Each mosaic pixel of a combined map is 
first determined to be either true or false by performing a binary AND operation 
on the respective mosaic pixels in the color gradient map and the intensity 

15 map. Those resulting mosaic pixels whose value is true are areas that combine 
smoothness and brightness. 

After performing the logical AND operation, a process known as 
morphological erosion is performed on the resulting map. Morphological 
erosion removes small true areas surrounding predominantly true areas in the 

20 following manner. For each mosaic pixel that has a value of true, examine all 
the neighboring mosaic pixels. If any of the neighboring mosaic pixels have a 
value of false, then replace the value of the central mosaic pixel with the value 
false. When this operation is performed a single time for all originally true 
pixels, the result is a map wherein each true area has had its peripheral area 

25 eroded. Sufficiently small true areas are removed entirely. 

Figure 5 shows the resulting combination map formed by the AND 
operation performed on Figure 3 and Figure 4, followed by one pass of the 
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morphological erosion process. The white area 510 shows where the value is 
true, signifying candidate face locations. The black area 520 shows areas 
rejected from further consideration as face locations. 

Referring now to Figure 6, a hue spectrum chart is shown, according to 
5 one embodiment of the present invention. Human skin color has been used as 
an important cue to locate human face in color images, based on the research 
results that human skin color tends to cluster in a pair of compact regions in 
certain transformed color space. One such transformed color space is hue 
saturation intensity (HSI) space. The chart of Figure 6 shows a range in HSI 
10 space. In HSI space, region 610 is reddish-green, regions 620, 630 are 

greenish, region 640 is bluish, region 650 is purplish, and region 660 is bluish- 
red. 

Referring now to Figure 7, a set of candidate patches and determined 
ellipses for the image of Figure 2 is shown, according to one embodiment of the 

15 present invention. The image of Figure 2 is first converted to HSI space, using 
commercially available software. The candidate patches of Figure 7 are then 
created by determining the parts of white area 510 of Figure 5 which have HSI 
space values of region 610 and region 660. The resulting candidate patches 
710, 720, 730, 740, and 750 are shown as irregular patches in Figure 7. 

20 Another step of morphological erosion may be performed on each 

candidate patch. As in the discussion in connection with Figure 5, each 
mosaic pixel in a candidate patch is examined to see if any neighboring mosaic 
pixels are not within the candidate patch. If so, then the examined mosaic 
pixel is removed from the candidate patch. 

25 Human facial outlines are consistently roughly elliptical. Therefore, for 

each candidate patch 710, 720, 730, 740, and 750 a corresponding ellipse 712, 
722, 732, 742, and 752, respectively, is determined. In one embodiment, a 
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Hoteling transform is used to create the corresponding ellipses. Commercially 
available Hoteling transform software may be used. In this step, the principle 
axis (major axis and minor axis) for each candidate patch are determined. The 
ellipse is completely specified by the principle axis. 
5 Next the degree of fit of the ellipse is measured. In one embodiment, the 

degree of fit may be measured by counting the number of mosaic pixels of the 
candidate patch that are within the boundaries of the corresponding ellipse. 
The ratio of the number of mosaic pixels of the candidate patch within the 
corresponding ellipse to the total number of mosaic pixels in the candidate 

10 patch may be used as a measure of degree of fit. In one embodiment, if the 
above ratio is above 80%, the degree of fit is said to be good; if the above ratio 
is below 80%, the degree of fit is said to be bad. In alternate embodiments, 
other measurements for good and bad fit may be used. 

Referring now to Figure 8, the face candidate map is superimposed on 

15 the image of Figure 2, according to one embodiment of the present invention. 
Putting the face candidate map of Figure 7 on the top of the original color 
image of Figure 2 yields Figure 8 which shows the detection initial result. 

Candidate patches with bad degrees of fit may be further subdivided. 
For example, candidate patch 720 includes faces 202 and 204. In order to 

20 further subdivide candidate patch 720, the above process of deriving the color 
gradient map and intensity map are repeated within the candidate patch. A 
new normalization for thresholding may be performed. New candidate patches 
may be identified, and each of these may have a new ellipse calculated. If the 
new ellipses are again bad fits, then the process of subdividing may be 

25 repeated as necessary. 

Whether or not the candidate patches are further subdivided, they are 
finally examined for a lack of detail. Human faces have great variety within the 
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overall shape because of the features, which may include the eyes, nose, and 
mouth. Each candidate patch may therefore be examined by calculating the 
standard deviation of intensity within each candidate patch. If the intensity 
within the candidate patch exceeds a standard deviation from the mean value 
5 at certain places, the candidate patch is considered sufficiently "face like". If, 
however, the intensity fails to exceed a standard deviation from the mean 
value, then the candidate patch is removed from further consideration. As an 
example, the star shape at candidate patch 710 fails this particular test, so it is 
removed from further consideration. 

10 Referring now to Figure 9, the final result of the process is shown, 

according to one embodiment of the present invention. Comparing previous 
and final results, we can see that the largest candidate patch 720 has been 
split into two candidate patches 924, 920, and the golden star at candidate 
patch 710 has been filtered out. The process has successfully and efficiently 

15 identified faces 202, 204, 206, and 208 of Figure 2. Face 210 was missed due 
to lack of intensity and color gradient changes. Hand 222 was mistakenly 
identified as a face. 

Referring now to Figure 10, a flowchart of process steps is shown, 
according to one embodiment of the present invention. The process begins at 

20 start 1000 with the selection of a color graphics image. In two parallel steps, 
smoothness detection 1010 and brightness detection 1020, the color gradient 
map (discussed in connection with Figure 3 above) and the intensity map 
(discussed in connection with Figure 4 above) are derived. Then in the binary 
AND 1030 step, the color gradient map and the intensity map are combined. 

25 In step 1032 a step of morphological erosion is performed. Then in step 

1034 a hue detection step is performed, yielding initial candidate patches. 
After a second step of morphological erosion 1036 is performed, the ellipses for 



80398P322 



13 



each candidate patch are determined in an ellipse fitting 1038 step. Then in 
step 1040, it is determined whether or not the ellipses are a good fit. If not, 
then the candidate patch is subjected to a further re-evaluation step 1042. 
After a further morphological erosion step 1044, a subsequent step of ellipse 
5 fitting 1046 is performed. These subsequent ellipses are again determined to 
be a good fit or a bad fit in step 1040. 

If, however, the ellipses are determined to be a good fit to the candidate 
patches in step 1040, then a subsequent determination is made whether the 
candidate patch is too smooth (e.g. lacks detail) in step 1050. Step 1050 may 

10 include the variation by more than a standard deviation approach used in 

connection with Figure 8 above. If the candidate patch is deteraiined to be too 
smooth to be a face, then in step 1054 the candidate patch is considered a bad 
patch and discarded. If, however, the candidate patch is determined to contain 
suitable variation, then in step 1052 the candidate patch is added to the list of 

15 good candidate patches. The Figure 10 process then ends 1060 with a list of 
good candidate patches. 

In the foregoing specification, the invention has been described with 
reference to specific embodiments thereof. It will however be evident that 
various modifications and changes can be made thereto without departing from 

20 the broader spirit and scope of the invention as set forth in the appended 

claims. The specification and drawings are, accordingly, to be regarded in an 
illustrative rather than a restrictive sense. Therefore, the scope of the 
invention should be limited only by the appended claims. 
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CLAIMS 



What is claimed is: 

1 y A method of determining at least one candidate patch for human 

2 / faces in a color graphic image, comprising: 

3 determining a first area wherein a color gradient has a low value; 

4 determining a second area wherein an intensity value has a high value; 

5 performing a logical AND on said first area and said second area to 

6 create a third area; and 

7 selecting portions of said third area with suitable hue saturation to form 

8 said at least one candidate patch. 

1 2. The method of claim 1, wherein said determining said first area 

2 uses a first threshold value comparison. 

1 3. The method of claim 2, wherein said first threshold value is 

2 determined by normalization. 

1 4. The method of claim 1, wherein said determining said second area 

2 uses a second threshold value comparison. 

1 5. The method of claim 4, wherein said second threshold is 

2 determined by normalization. 

1 6. The method of claim 1, further comprising eroding said third area. 

1 7. The method of claim 6, wherein said eroding is morphological. 
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1 8. The method of claim 1, further comprising fitting an ellipse to one 

2 of said at least one candidate patch. 

1 9. The method of claim 8, further comprising determining if said 

2 ellipse is a bad fit to said one of said at least one candidate patch. 

1 10. The method of claim 9, further processing said one of said at least 

2 one candidate patch when said ellipse is a bad fit. 

1 11. The method of claim 10, further comprising determining if said one 

2 of said at least one candidate patch is too smooth. 

1 J^L. A system configured to determine at least one location of a 

2 human face in a color graphic image, comprising: 

3 a color gradient map configured to indicate true where a color gradient 

4 has a low value; 

5 an intensity map configured to indicate true where an intensity value has 

6 a high value; 

7 a combined map configured to indicate true where said color gradient 

8 map is true and said intensity map is true; and 

9 at least one candidate patch selected from said combined map, wherein 
10 said candidate patches each have suitable hue saturation. 

1 13. The system of claim 12, wherein said color gradient map includes a 

2 first threshold. 
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1 14. The system of claim 13, wherein said first threshold is determined 

2 by normalization. 

1 15. The system of claim 12, wherein said intensity map includes a 

2 second threshold. 

1 16. The system of claim 15, wherein said second threshold is 

2 detemiined by normalization. 

1 17. The system of claim 12, wherein said combined map includes an 

^ 2 eroded boundary. 

fi 1 18. The system of claim 17, wherein said boundary is morphologically 

O 2 eroded. 

Q 1 19. The system of claim 12, further comprising an ellipse fitted to said 

G 2 at least one candidate patch. 

1 20. The system of claim 19, wherein said ellipse includes a degree of fit 

2 measure. 

1 21. The system of claim 20, wherein said at lease one candidate patch 

2 is marked for further processing when said degree of fit is bad. 

1 22. The system of claim 21, further comprising a candidate patch 

2 examiner configured to determine whether said at least one candidate patch is 

3 too smooth. 



1 23. A machine-readable medium having stored thereon instructions for 

2 processing elements, which when executed by said processing elements 

3 perform the following: 

4 determining a first area wherein a color gradient has a low value; 

5 determining a second area wherein an intensity value has a high value; 

6 performing a logical AND on said first area and said second area to 

7 create a third area; and 

8 selecting portions of said third area with suitable hue saturation to form 

9 at least one candidate patch. 
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ABSTRACT 



A system and method for determining the location of human faces within 
a color graphics image is disclosed. The proposed method consists of several 
steps to distinguish face candidates from a complex background. The method 
determines areas with both low color gradient values and high relative 
intensity. These areas are then further selected on the basis of hue saturation. 
A final series of steps determines which of these candidate areas represent 
human faces within the original image. 
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I hereby state that I have reviewed and understand the contents of the above-identified 
specification, including the claim(s), as amended by any amendment referred to above. I do not 
know and do not believe that the claimed invention was ever known or used in the United States of 
America before my invention thereof, or patented or described in any printed publication in any 
country before my invention thereof or more than one year prior to this application, that the same 
was not in public use or on sale in the United States of America more than one year prior to this 
application, and that the invention has not been patented or made the subject of an inventor's 
certificate issued before the date of this application in any country foreign to the United States of 
America on an application filed by me or my legal representatives or assigns more than twelve 
months (for a utility patent application) or six months (for a design patent application) prior to this 
application. 

I acknowledge the duty to disclose all information known to me to be material to patentability as 
defined in Title 37, Code of Federal Regulations, Section 1 .56. 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 1 19(a)-(d), of any 
foreign application(s) for patent or inventor's certificate listed below and have also identified below 
any foreign application for patent or inventor's certificate having a filing date before that of the 
application on which priority is claimed: 

Priority 

Prior Foreign Application(s) Claimed 



(Number) 


(Country) 


(Day/Month/Year Filed) 


Yes 


(Number) 


(Country) 


(Day/Month/Year Filed) 


Yes 



(Number) (Country) (Day/Month/Year Filed) Yes No 
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I hereby claim the benefit under title 35, United States Code, Section 1 19(e) of any United States 
provisional application(s) listed below: 



(Application Number) Filing Date 



(Application Number) Filing Date 



I hereby claim the benefit under Title 35, United States Code, Section 120 of any United States 
application(s) listed below and, insofar as the subject matter of each of the claims of this application 
is not disclosed in the prior United States application in the manner provided by the first paragraph 
of Title 35, United States Code, Section 1 12, 1 acknowledge the duty to disclose all information 
known to me to be material to patentability as defined in Title 37, Code of Federal Regulations, 
Section 1 .56 which became available between the filing date of the prior application and the national 
or PCT international filing date of this application: 



(Application Number) Filing Date (Status - patented, 

pending, abandoned) 



(Application Number) Filing Date (Status -- patented, 

pending, abandoned) 

I hereby appoint the persons listed on Appendix A hereto (which is incorporated by reference and a 
part of this document) as my respective patent attorneys and patent agents, with full power of 
substitution and revocation, to prosecute this application and to transact all business in the Patent 
and Trademark Office connected herewith. 

Send correspondence to Dennis A. Nicholls , BLAKELY, SOKOLOFF, TAYLOR & 

(Name of Attorney or Agent) 
ZAFMAN LLP, 12400 Wilshire Boulevard 7th Floor, Los Angeles, California 90025 and direct 

telephone calls to Dennis A. Nicholls , (408) 720-8300. 

(Name of Attorney or Agent) 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made 
are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United 
States Code and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 

Full Name of Sole/First Inventor Lizhi Wang 

Inventors Signature ^gffi" L^^^^ Date &f 

Residence Cupertino. California Citizenship Peoples Republic of China 

(City, State) (Country) 

Post Office Address 21025 Cory Court 

Cupertino. California 95014 
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APPENDIX A 



William E. Alford, Reg. No. 37,764; Farzad E. Amini, Reg. No. P42,261; Aloysius T. C. AuYeung, Reg. No. 
35,432; William Thomas Babbitt, Reg. No. 39,591; Carol F. Barry, Reg. No. 41,600; Jordan Michael 
Becker, Reg. No. 39,602; Bradley J. Bereznak, Reg. No. 33,474; Michael A. Bernadicou, Reg. No. 35,934; 
Roger W. Blakely, Jr., Reg. No. 25,831 ; Gregory D. Caldwell, Reg. No. 39,926; Ronald C. Card, Reg. No. 
44,587; Andrew C. Chen, Reg. No. 43,544; Thomas M. Coester, Reg. No. 39,637; Alin Corie, Reg. No. 
P46,244; Dennis M. deGuzman, Reg. No. 41 ,702; Stephen M. De Klerk, under 37 C.F.R. § 10.9(b); 
Michael Anthony DeSanctis, Reg. No. 39,957; Daniel M. De Vos, Reg. No. 37,813; Robert Andrew Diehl, 
Reg. No. 40,992; Sanjeet Dutta, Reg. No. P46,145; Matthew C. Fagan, Reg. No. 37,542; Tarek N. Fahmi, 
Reg. No. 41,402; Paramita Ghosh, Reg. No. 42,806; James Y. Go, Reg. No. 40,621; James A. Henry, 
Reg. No. 41,064; Willmore F. Holbrow III, Reg. No. P41,845; Sheryl Sue Holloway, Reg. No. 37,850; 
George W Hoover II, Reg. No. 32,992; Eric S. Hyman, Reg. No. 30,139; William W. Kidd, Reg. No. 
31 ,772; Sang Hui Kim, Reg. No. 40,450; Eric T. King, Reg. No. 44,188; Erica W. Kuo, Reg. No. 42,775; 
Kurt P. Leyendecker, Reg. No. 42,799; Michael J. Mallie, Reg. No. 36,591 ; Andre L. Marais, under 37 
C.F.R. § 10.9(b); Paul A. Mendonsa, Reg. No. 42,879; Darren J. Milliken, Reg. 42,004; Lisa A, Norris, 
Reg. No. 44,976; Chun M. Ng, Reg. No. 36,878; Thien T. Nguyen, Reg. No. 43,835; Thinh V. Nguyen, 
Reg. No. 42,034; Dennis A. Nicholls, Reg. No. 42,036; Daniel E. Ovanezian, Reg. No. 41,236; Marina 
Portnova, Reg. No. P45,750; Babak Redjaian, Reg. No. 42,096; William F. Ryann, Reg. 44,313; James 
H. Salter, Reg. No. 35,668; William W. Schaal, Reg. No. 39,018; James C. Scheller, Reg. No. 31,195; 
Jeffrey Sam Smith, Reg. No. 39,377; Maria McCormack Sobrino, Reg. No. 31 ,639; Stanley W. Sokoloff, 
Reg. No. 25,128; Judith A. Szepesi, Reg. No. 39,393; Vincent P. Tassinari, Reg. No. 42,179; Edwin H. 
Taylor, Reg. No. 25,129; John F. Travis, Reg. No. 43,203; George G. C. Tseng, Reg. No. 41 ,355; Joseph 
A. Twarowski, Reg. No. 42,191; Lester J. Vincent, Reg. No. 31,460; Glenn E. Von Tersch, Reg. No. 
41,364; John Patrick Ward, Reg. No. 40,216; Mark L. Watson, Reg. No. P46,322; Thomas C. Webster, 
Reg. No. P46,154; Charles T. J. Weigell, Reg. No. 43,398; Kirk D. Williams, Reg. No. 42,229; James M. 
Wu, Reg. No. 45,241; Steven D. Yates, Reg. No. 42,242; and Norman Zafman, Reg. No. 26,250; my 
patent attorneys, and Justin M. Dillon, Reg. No. 42,486; my patent agent, of BLAKELY, SOKOLOFF, 
TAYLOR & ZAFMAN LLP, with offices located at 12400 Wilshire Boulevard, 7th Floor, Los Angeles, 
California 90025, telephone (310) 207-3800, and James R. Thein, Reg. No. 31 ,710, my patent attorney. 
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APPENDIX B 



Title 37, Code of Federal Regulations, Section 1 .56 
Duty to Disclose Information Material to Patentability 



(a) A patent by its very nature is affected with a public interest. The public interest is best served, 
and the most effective patent examination occurs when, at the time an application is being examined, the 
Office is aware of and evaluates the teachings of all information material to patentability. Each individual 
associated with the filing and prosecution of a patent application has a duty of candor and good faith in 
dealing with the Office, which includes a duty to disclose to the Office all information known to that individual 
to be material to patentability as defined in this section. The duty to disclosure information exists with respect 
to each pending claim until the claim is cancelled or withdrawn from consideration, or the application becomes 
abandoned. Information material to the patentability of a claim that is cancelled or withdrawn from 
consideration need not be submitted if the information is not material to the patentability of any claim 
remaining under consideration in the application. There is no duty to submit information which is not material 
to the patentability of any existing claim. The duty to disclosure all information known to be material to 
patentability is deemed to be satisfied if all information known to be material to patentability of any claim 
issued in a patent was cited by the Office or submitted to the Office in the manner prescribed by §§1 .97(b)-(d) 
and 1 .98. However, no patent will be granted on an application in connection with which fraud on the Office 
was practiced or attempted or the duty of disclosure was violated through bad faith or intentional misconduct. 
The Office encourages applicants to carefully examine: 

(1 ) Prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) The closest information over which individuals associated with the filing or prosecution of a 
patent application believe any pending claim patentably defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information is material to patentability when it is not cumulative to 
information already of record or being made or record in the application, and 

(1 ) It establishes, by itself or in combination with other information, a prima facie case of 
unpatentability of a claim; or 



(ii) Asserting an argument of patentability. 

A prima facie case of unpatentability is established when the information compels a conclusion that a claim is 
unpatentable under the preponderance of evidence, burden-of-proof standard, giving each term in the claim 
its broadest reasonable construction consistent with the specification, and before any consideration is given to 
evidence which may be submitted in an attempt to establish a contrary conclusion of patentability. 

(c) Individuals associated with the filing or prosecution of a patent application within the 
meaning of this section are: 

(1) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the 
application and who is associated with the inventor, with the assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by 
disclosing information to the attorney, agent, or inventor. 



(2) 



It refutes, or is inconsistent with, a position the applicant takes in: 



(i) 



Opposing an argument of unpatentability relied on by the Office, or 
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